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Methods and Apparatus for Making Images Including Depth Information 

This invention relates to making images including depth information, which is to say, 
primarily, the production of an image of an object which includes information about the 
distance from the viewer of an image of parts of the imaged object. 

Images including depth information include: 

mask images, produced from a single viewpoint; 

angular-composite images, produced from two or more viewpoints differing in 
angular orientation of the object about a single axis; 

fully three dimensional images, produced from three or more viewpoints differing 
1 5 in angular orientation of the object about at least two orthogonal axes. 

A three-dimensional representation of any of those images of, say, a human head could 
be, for example, a sculpture, or a rendering in glass or clear plastic of the shape of the 
head by laser-produced point strains, visible as bright points under Ulumination. 

20 However, a two-dimensional representation of any of those images, for example, one 
displayed on a video screen, can have image depth information which can be perceived 
as by manipulating the image, e.g. by rotation, or if it can be viewed by an arrangement 
such as a decoding screen, in the case of integral imaging, or by separating two two- 
dimensional images taken from adjacent vantage points, one into each eye, simulating 

25 binocular vision. 6 

The term "depth imaging", as used herein, means the production of an image with depth 
^formation, whether or not actually displayed, but at least with the potential of being 
™ + d u lspla y? d or . used t0 P roduce something that can be viewed as a two-dimensional or 
30 three-dimensional representation of an object, and includes, therefore, the process of 
capturing [information, including depth information, about the object, and the processing 
ot that information to the point where it can be used to produce an image. 

One method for depth imaging, disclosed in US-A-4657394, involves illuminating an 
object with a beam of light having a sinusoidally varying intensity pattern, produced by a 
grating. This throws a pattern of parallel light and dark stripes on to the object When 
viewed from an offset position, the stripes are deformed. A series of images is formed 
using a linear array camera, as the object is rotated. Each image will be different, and 
from the different images, the position, in three dimensions, of each point on the surface 
of the object is calculated by triangulation, according to an algorithm programmed into a 
computer. 
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Other methods ^^ determination using triangulation from multiple images are 
disclosed mDE-A-19515949,DE-A-4416108,JP-A-4416108 and US-A-5085502 
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Such methods involve expensive equipment, are difficult to carry out and take a long 
time - usually about an hour. 

The present invention provides methods that are much faster and which use less 
expensive equipment, and which, in particular, are capable of being used in connection 
with personal computers as a desktop depth imaging facility. 

The invention comprises a method for making an image of an object including depth 
information comprising the steps of: 

• illuminating the object with a periodic pattern of light from an 
illuminating arrangement; 

• the illuminating arrangement being such that the pattern is in focus in a 
focal plane and defocuses progressively away from said focal plane; 

• the object being placed such that different parts of it are at different 
distances from the focal plane; 

• capturing image data from the thus-illuminated object; 

• analysing the captured image data to extract depth information based on 
the extent of defocussing of the pattern; and 

displaying an image of the object without the pattern and with depth information. 

The image may be a mask image. The image data may be captured in a single image 
The image may be an angular-composite image, and the data may then be captured in at 
least two mask images differing in the angular orientation of the object about a single 
axis orthogonal top a line between the object and the illuminating arrangement. 

The image may be a 3D image. The image data may than be captured in at least three 
mask images differing in the angular orientation of the object about at least two axes 
orthogonal to a line joining the object and the illuminating arrangement. 

The object may be placed such that it does not intersect the focal plane, and may be 
placed such that it is in a region in which rate of change of defocussing with distance 
from the illuminating arrangement is greatest, and/or a region in which the rate of change 
of defocussing with distance from the illuminating arrangement is reasonably constant. 

The pattern may be removed from the image by capturing image data corresponding to 
wimoufui e piuSn PatteniS ^ ^ imag6 ^ fr ° m ° bjeCt illumina ted 

The pattern may be of alternating bright and dark lines; it is desirable that no region of 
the pattern on the object is completely unilluminated - essentially no information can be 
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gathered from unffluminated regions - and, of course, it is desirable that no substantial 
part of the object should be totally absorbing. 

The pattern may be generated by a grating, which may be of equally spaced light and 
5 dark parallel lines. 

The concept of projecting an image of a grating onto a 3D object to produce a composite 
image is known in the field of 3D measurement using structured light. Here the shape of 
the 3D object deforms the grating in such a way that the shape may be calculated using 
10 triangulation methods (for example - WO 00/70303). Such methods require the imaging 
device to be positioned at an angle to the projection device. In such measurements, the 
deformation of the grating makes grating removal difficult, as a loss in the periodicity of 
the grating has occurred. Thus depth is recovered but texture mapping requires an image 
without the grating present. 

15 

The projection of a grid image onto an object is known also in the art of confocal 
microscopy. Here the grating has only a narrow depth of focus and the presence of the 
grating image serves to locate the depth of those parts of the object, which lie in the same 
focal plane as the grating image (for example - WO 98/45745). Here the grid is removed 

20 by a phase stepping method. In brief, the technique requires at least three phase-stepped 
composite images and the mathematical treatment is simplified if the phase stepping is 
set at 120 degrees. A second example (DE 199 30 816) uses a similar phase stepping 
method; in this case four steps are used at 90-degree intervals. In practice it is possible to 
perform an approximate phase stepping method using just two steps. In this case parts of 

25 the grating image m parts of the composite image may not removed completely. 

In addition to phase-stepping, correlation methods may be used to subtract a grating 
image from a composite image. The use of correlation functions in the statistical analysis 
of signals and images is widespread. The exact nature of the correlation analysis is 
30 dependant on the image data available, in particular: 

1 . knowledge of the form of the grating image, e.g. sine wave 

2. knowledge of the period and amplitude of the grating image 

3. knowledge of the position of the function in the composite image 

4. knowledge of the wide field image, i.e. image in the absence of the grating 
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Where both grating and wide field images are known, the grating may be removed 
completely and depth information may be gained at the pixel level Where less 
information is available, it may be necessary to recover depth and texture information at 
the period level. 

The extent of defocussing may be calculated on the basis of the width of a line of the 
pattern or on the basis of the modulation contrast of the pattern. 

The frequency response of a defocused optical system was first described by H H 
Hopkins (Proc Roy. Soc. A 231, 3,1955). Here a description is given of the defocu^ 
function and its dependence on the image and optical properties. In brief, the distribution 
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of intensity in the image plane is found by integrating the intensity distributions in the 
diffraction images associated with each point in the object. For a simple object (a lined 
grating) the defocus function (D) (also termed the optical transfer function and the 
modular transform function) may be calculated analytically and is often expressed in 
terms of a universal frequency function (s). By definition, V is inversely proportional to 
the aperture of the lens and proportional to the spacing of the grating. In practice this is 
seen as fine structure exhibiting only a short depth of focus whereas small apertures give 
a large depth of focus. F B 

With knowledge of the basic optical parameters, D(s) versus s may be plotted for 
individua optical systems. The function is seen to display a largely linear region 
between tiie values 0 8 and 0.2. This is advantageous when depth distance is to be 
calculated from the defocus function. 

^ h ^??Qfo\ 0n T? f the u de 5T S fcncti0n is ^ ven ^ P A - Stokse * (J- Opt- Soc. Am. 
59M>, 1314 1969). Here the defocus function is calculated analytically using both 

ddfracbon and geometrical optics theories. In addition, an empirical treatise is given. 

The defocus function is shown to asymmetrical either side of the focal plane (sphere) 

with a longer depth of defocus being observed behind the plane of focus. 

t^!TJ?*™ * he J Cam tf ^ Parallel scan lines, parallel to or angled with respect to 
p^tem ^ P SC3n lbleS may be at rfght 8118168 t0 *" ^es of the 

b?pixeTbasTs age ^ COmpriSe pixel image data ' which ma y be ^alysed on a pixel 

Image . capture may be by a line scan camera or by an area scan camera, and may be in 
ZaZl ™«° r C ?u°T- I he CaptUr6d image data ™y be analysed to calculate coloS 
tte pTtttrn g ^ ° f ^ imag6 ' namdy fr0m bri 8 htness P eaks of 

— ? fJ G ^jf?™**™ ™* be adjusted using a calibration, as by a calibration 
^-up table, which may be generated by comparing calculated with actual depth 
measurements on a specimen object. p 

The image may be formatted for display using any preferred display system such for 
example, as a video screen driven by software simulating and mampulatSb^ages 
or as an integral or multiview image which can be viewed using a decodmglreen * 

~ ^ m image ° f ° bjeCt 
* Stte^hgSf adapt6d to illuminat « *e object with a periodic 
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• the illuminating arrangement being such that the pattern is in focus in a focal 
plane and defocuses progressively away from said focal plane; 

• the object being locatable with respect to the illuminating arrangement such that 
5 different parts of it are at different distances from the focal plane; 

• image data capturing means adapted to capture image data from the thus 
illuminated object; 

10 • depth analysis means adapted to analyse captured image data to extract depth 

information based on the extent of defocussing of the pattern; and 

• image display means for displaying an image of the object without the pattern and 
with depth information. 

The image data capturing means may capture a mask image, and may comprise a one- 
dimensional or a two-dimensional array of detectors. Such may comprise a monochrome 
or colour CCD or CMOS camera. 

20 The illuminating arrangement may comprise a light source, focussing means and a 
grating. 

The light source may comprise a source of incoherent light, such as an incandescent 
filament lamp, a quartz-halogen lamp, a fluorescent lamp or a light-emitting diode. The 
25 light source may, however, be a source of coherent light, such as a laser. 

The focussing means may comprise a lens or a mirror, and may comprise a cylindrical, 
spherical or parabolic focussing arrangement. 

30 The imaging apparatus may comprise a support for an object to be imaged. The support 
may also support the illuminating arrangement in such relationship that the object is 
supported so that the focal plane does not intersect the object, and desirably in a region in 
which the rate of change of defocussing with distance from the illuminating arrangement 
is reasonably constant. 

The support may also permit relative adjustment between the object and the illuminating 
arrangement, and may comprise a turntable. 

The apparatus may also comprise means adapted to vary the periodic pattern of light, 
40 which may comprise means adapted to alter the orientation of a grating producing a 
periodic pattern of light. 
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The image display means may comprise a video screen driven by software capable of 
simulating and manipulating a 3D image. 
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Embodiments of imaging apparatus and methods of imaging according to the invention 
will now be described with reference to the accompanying drawings, in which: 

Figure 1 shows (a) a mask image view of an object O from a single viewpoint; (b) a 
peripheral view such as will, when integrated, give rise to an angular- 
composite image; and (c) a fully three-dimensional view in which the 
object is rotated with respect to the viewer about two orthogonal axes; 

Figure 2 illustrates the underlying principle of progressive defocussing with depth; 

Figure 3 is a view of a first embodiment of apparatus, for mask or angular- 
composite imaging; 

Figure 4 is a view of a second embodiment of apparatus, for fully three- 
15 dimensional imaging ' 

Figure 5 illustrates four embodiments (a) - (d) of an illuminating arrangement; 

Figure 6 is a flow diagram showing an overview of the imaging method; 

20 

Figure 7 is a flow diagram showing in detail one embodiment of one step in the 
flow diagram of Figure 7; 

Figure 8 is a flow diagram showing in detail another embodiment of the step of 
25 Figure 8; 
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Figure 9 is a flow diagram showing in detail yet another embodiment of the step of 
Figure 8; * 

30 Figure 10 is a flow diagram showing in detail one embodiment of another step in the 
flow diagram of Figure 7; 

Figure 11 is a flow diagram showing in detail another embodiment of the step of 
Figure 11; y 

Figure 12 is a flow diagram showing in detail yet another embodiment of the step of 
Figure 11; v 

Figure 13 is a flow diagram showing a generalisation of the detail of Figure 13; 

Figure 14 is a flow diagram showing one complete measurement method: 

Figure 15 is a flow diagram showing another complete measurement method; 

45 Figure 16 is a flow diagram showing another complete measurement method; and 
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Figure 17 is a flow diagram showing a fourth complete measurement method. 

The drawings illustrate an imaging apparatus for making an image of an object O 
including depth information, comprising: 

5 

• an illuminating arrangement 1 1 adapted to illuminate the object O with a periodic 
pattern 12 of light; 



• the illuminating arrangement 11 being such that the pattern 12 is in focus in a 
10 focal plane 13 and defocuses progressively away from said focal plane 13; 

• the object O being locatable with respect to the illuminating arrangement 1 1 such 
that different parts of it are at different distances from the focal plane 13; 

15 • image data capturing means 14 adapted to capture image data from the thus 

illuminated object 11; 



20 



depth analysis means 15 adapted to analyse captured image data to extract depth 
information based on the extent of defocussing of the pattern 12; and 

image display means 16 for displaying an image 17 of the object O without the 
pattern 13 and with depth information. 



Figure 1 illustrates three different methods of imaging that can yield depth information 
25 about an object O. In Figure 1(a), the object is viewed from a single viewpoint. This is 
not usually conducive to capturing depth information, but, using the present invention, 
depth information can be extracted from such a view. An image thus formed is termed a 
mask image. In Figure 1(b), the object O is viewed from more than one viewpoint. In 
human binocular vision, and in binocular or multiview photography, depth information is 
30 gleaned from differences in the images. In integral imaging, a single viewpoint is 
apparently used, but a wide 'taking' aperture and integral optics afford many different 
viewpoints within the taking aperture. While such measures will serve to give depth 
information which can make an image appear to be three-dimensional, this will only 
apply to such regions of the object as are visible from the viewing position or positions. 
35 In order to acquire information about the back of the object, it is necessary to view from 
at least two, preferably more different directions. Such an image taken from two or more 
viewpoints as the object is rotated relatively to a single taking position is termed an 
angular-composite image. 

40 If the top and bottom of the object are to be imaged, it is necessary to have further 
viewpoints, with the object rotated, relative to the taking position, about two axes A, B 
each orthogonal to a line X joining the object O and the viewing position P, as shown in 
Figure 1(c). An image incorporating such information can be termed a fully three 
dimensional image. 

45 
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By and large, objects stand on the ground or a base, and so an underview is unnecessary, 
and sufficient information can be gleaned from an angular-composite image, which 
corresponds to human binocular vision, but which can contain more information if the 
back of the object is taken into account.. 

5 

Using methods as herein described, simple mask images, angular-composite images and 
fully three dimensional images can be made, each with depth information sufficient to 
produce a final image with the appearance of depth. 

10 Figure 2 illustrates the underlying principle. A light source L casts a pattern of light and 
dark lines from a grating Ml by means of a lens Fl. The pattern is in focus at a focal 
position f distant d from the lens Fl. Were the pattern to be cast on a screen closer than 
the distance d, the pattern would be out of focus, and is shown diagrammatically as being 
more out of focus the closer the screen approaches the lens Fl. Contrast between the 

15 light and dark lines of the pattern is greatest at the focal distance d, and falls off towards 
the lens Fl. The measured modulation depth of the pattern gives an indication of the 
distance of the screen from the focal position f. 

If, instead of a flat screen, the pattern falls on a shaped object, the pattern will be more or 
20 less out of focus at different positions on the object, and the modulation depth would be 
correspondingly different. The distance of each point of the object from the focal 
position can be calculated as a function of the measured modulation depth at that point. 
This will be termed "structured modulation imaging" (SMI). 

25 The method differs from triangulation methods, in that imaging and viewing can take 
place from a single position, and the pattern defocuses over the depth of the object, 
whereas in triangulation, sharp focus over the whole object is preferred. 

The modulation depth as a function of distance from a focal plane of a lens system is 
30 discussed in WO-A-98/45745 and DE 199 30 816 Al. 

In those publications, which are concerned with microscopy, it is taught that the grid may 
be displaced so that the pattern moves into discrete positions across the object displaced 
by fractions of the grating constant, and an image of the pattern's projection on the object 
35 is recorded for each position of the grating. Only the in-focus parts of each image are 
used; they are assembled into a single image. The modulation depth information is used 
to remove the pattern from the image mathematically. 

In contrast, the method according to the invention is concerned with macroscopic 
40 imaging, and does not depend on such displacement of the grid. 

The method comprises the steps of: 
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• illuminating the object O with a periodic pattern 12 of light from an 
illuminating arrangement 11; 
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• the illuminating arrangement 1 1 being such that the pattern 12 is in focus 
m a focal plane 13 and defocuses progressively away from said focal plane 
13j 

• the object O being placed such that different parts of it are at different 
distances from the focal plane 13; 

• capturing image data from the thus-illuminated object O; 

• analysing the captured image data to extract depth information based on 
the extent of defocussing of the pattern 12; and 

• displaying an image 17 of the object without the pattern 12 and with depth 
information. 

The image may be a mask image, in which the captured image data are captured in a 
single image, or it may be an angular-composite image, in which the image data are 
captured in at least two mask images differing in the angular orientation of the object O 
about a single axis orthogonal to a line between the object O and the illuminating 
arrangement 1 1. Or the image may be a 3D image, in which the image data are captured 
in at least three mask images differing in the angular orientation of the object about at 
least two axes orthogonal to a line joining the object O and the illuminating 
arrangement 1 1. e 

The method will be described in these three aspects with reference to the flow diagrams 
of Figures 6 to 17, and Figures 3, 4 and 5. 

Figure 3 shows apparatus for carrying out mask or angular-composite imagine 
comprising an illuminating arrangement 11, and a turntable 31 on which the object O is 
pkiced. The turntable 31 is rotated by an electric motor 32 about an axis 33 which is 
orthogonal to the optical axis 34 of the illuminating arrangement 11. The motor 32 is 
am^te a C ° mpUter 35 t0 r ° tate ^ stable stepwise through selected angular 

Figure 4 shows apparatus for carrying out fully three-dimensional imaging, as well of 
course as mask and angular-composite imaging. Similar to the embodiment of Figure 3 
it has however, a support 41 on the turntable supporting the object O on an axis 42 about 
which it can be rotated by a second electric motor 43, also controlled by the computer 3? 
also in desired angular steps. F JJ > 

In the apparatus of both Figure 3 and Figure 4 is an image capture arrangement 36 that 

S ZZ™:™^ ^ ° r SC3n digital Camera ^angement. A keyboard t 37 is 
used tro mput instructions into the computer 35, and a VDU 38 displays the image. 

Figure 5 shows four different embodiments of the illuminating arrangement 11. 
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Figure 5(a) shows a light source L such as an incandescent filament lamp, illuminating a 
parallel line grating Ml with a focussing arrangement Fl, such as a convex lens forming 
a virtual image of the grating in a focal plane P. The grating Ml can be mounted on a 
carnage (not shown), which would also be controlled by the computer 35 of Figures 3 or 
5 4, to move in the direction of Arrow A perpendicular to the rulings of the grating Ml . 

Figure 5(b) shows a slit D interposed between the grating Ml and focussing means Fl of 
Figure 5(a). The grating Ml can be moved, again by the carriage, not shown, angularly 
with respect to the slit D and also perpendicularly to the lines of the grating Ml, Arrows 
10 B and A. These movements alter the spatial frequency of the illumination pattern 
allowing altered modulation contrast characteristics for a fixed focussing means Fl . 

Figure 5(c) shows a helical grating M3 and a slit D placed between the light source L and 
the focussing means Fl . The light source L here can be a fluorescent tube. Rotation of 
1 5 the helical grating about its axis moves the pattern projected on the object O. 

Figure 5(d) shows a collimated, controlled intensity light source L projecting on to a 
scanning mirror 51 which, at any one position, projects a strip of illumination on to the 
object O. If the intensity of the light source is synchronised with the scan, any desired 
20 light intensity pattern can be displayed on the object O. 

Figure 6 is a flow diagram generic to all methods for forming and displaying images with 
depth information. 



25 To begin the process at Step 1, the object O is placed in the apparatus, on the 
turntable 31, and illuminated with whichever pattern is desired for the image in question. 

The object can be of any shape, size (so long as it fits into the apparatus) and colour, the 
only limitation being that it must reflect light at least to some extent, so it cannot be black 
30 or totally absorbing over its entire surface. It should also preferably not be totally 
transparent Objects with black regions or of glass or transparent plastics materials will 
give poor depth resolution. Objects up to 150mm long can be imaged in an apparatus 
with a paper size A4 footprint, which will conveniently fit on a desktop. 

35 The software provides at Step 2 an option to customise the measurement parameters and 
set the customised parameters before capturing the image in the camera 35. Such 
customisation can include selection of: 



• colour, monochrome or sepia 

• grid defocus over radius or diameter of turntable 

• grid frequency 

• lamp intensity 

• colour and polarising filters 

• camera lens aperture setting 

f automatic gain control (AGC) on camera 

• gamma setting on camera 
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brightness on camera 
contrast on camera 

use of RBG channels separately or combined in depth calculation 
number of pixels, horizontal and vertical, used on camera 
number of steps per rotation (for angular-composite and 3D images) 
number of rotations of turntable 

number of steps per period, i.e. how many grids are to be used in the algorithm 
grid divergence corrections 

averaging algorithms, and at which stage in the calculations they are used 
smoothing algorithms, and at which stage in the calculations they are used 
texture map algorithm 
geometry transformation algorithm 
3D viewer 

15 After the image is captured at Step 3, it is subjected, at Step 5, to general image 
processing, involving, for example, the use of smoothing algorithms and cut and 
reassembly operations. 

The processed image is then further processed at Step 6 to extract the depth information 
20 This will be dealt with in detail below. 

The image information yielded by Step 6 is then further processed at Step 7 to add colour 
and or texture, as will, again, be further discussed below. 

25 At Step 8, geometrical mapping is performed, which might involve changing the 
coordmate system from cartesian coordinates, in which the initial measurement might 
have been made, to cylindrical coordinates, in which the final image might be displayed. 

Finally, at Step 9, the image is displayed on whatever display arrangement has been 
30 selected to display it. This might be a computer monitor screen, which will, of course 
display only a 2D image, but such image can be manipulated by rotating it, for example' 
to show it from different aspects, and even show the back of the imaged object Or it 
might be a monitor screen with a decoding screen, the image on the screen having been 
processed into the format of an integral image such that, viewed through the decoding 
screen, the image appears to have depth appropriate to binocular vision. Or the image 
information might be used to generate a true 3D set of coordinates used to drive a laser to 
write a 3D image in a glass or transparent plastic block. 

In Step 4 as seen in Figure 6, the object is moved, unless a single mask image is to be 
made The movement will be, in the case of an angular-composite image, a rotation 
about the axis 33 of the turntable. In this case, the illumination, and the image, will be of 
a vertical strip, as seen in Figure 5(d), and the turntable will be stepped around so that the 
entire object (or so much of it as may be desired to image) is imaged in vertical strips. 
Such strips are welded' together in the general image processing step, Step 5. If fully 
3D image is required, the ruuTuun about the axis 42 of the turntable 31 is also effected 
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Possibly, the object O is first imaged as an angular-composite image when it is the right 
way up, then it is flipped through 90° about axis 42 and another set of images made. 

Figure 7 is a sub-flow diagram of the operation of making a mask image, i.e. one made as 
from a smgle viewpoint without rotation of the object. The whole of the object area 
facing the imaging apparatus is illuminated with the pattern. 

There are four possible routes through this sub-flow diagram. 

Route 1 is the simplest. First, the image is captured - this may be repeated one or more 
times, to gam better resolution from averaging multiple images. The single or single 
averaged image is then sent straight to step 5 for general image processing. The image 
will, of course, contain depth information, in the form of the extent of defocussing of the 
pattern at different locations on the image, manifest as modulation contrast. In the 
subsequent image processing, this information is extracted and the pattern removed bv 
appropriate algorithms. y y 

On Route 2 a first image is made with the grid pattern in place, than a second image is 
made with the grid moved out of the way. Both first and second images, of course, may 
be made more than once and averaged. Both images are sent for further processing, 
depth information bemg extracted from the first image, and transferred to the second 
image, which does not, of course, have the pattern, so there is now no need of a pattern 
removal operation. vaucw 

0 L R0U \ 3, l he 8ri J ?' ° n R ° Ute 4 ' the object < which a™™* t0 the same thing) is moved 
a known fraction of a grid period, and a second image taken. These two images are then 
sent for processing to extract depth information and remove the pattern for the final 
image processing steps. 

Figure 8 is a sub-flow diagram for Step 4 for an angular-composite image. A first image 

rJSTt ' lf de r d ' 35 bef ° re ' ° ne ° r more re P eat ca P toes ^ The objeTS 
rotated a known angular extent, and another image is made. This is repeated until the 
whole object, or such part of it as is required, has been imaged in vertical s«ps « 
explained above. A composite image is built up from the multiple strip image?* tS 
SwlZ ge t Pr ° CeSSmg f P ; StCp 5 - In Ms °P eration ' the P*«™ ™y be shifted! 

sssx r; M t ly ' or to move its or *• object ' a fraction of Isdd period: 

SSSon ^T 3 SUb " fl T diagram f ° r St6p 4 for a three-dimensional imaging 
SS£ i ♦ P/ ocedure » •» ^ Step 4 for the angular-composite image, with th? 
addmonal step of moving the object relatively to the camera,about the ot^X 

Figure 10 is a sub-flow diagram for Step 6 for the single image, single grid method 
Route 1 of sub-flow diagram, Figure 6. The single image is taken from L SndSS 
processing step, Step 5 and the pixel brightness values read into an image arf ay^on S 
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further signal processing may be carried out if desired. The array dimensions are 
calculated and the length and number of periods of the pattern are calculated. The 
processing may be carried out on a period or pixel basis. On a period basis, the 
maximum, minimum and mean pixel brightness values are calculated for each period in 
5 each line of the array. In pixel based processing, the pixel phase and amplitude are 
calculated for each line of the array. Colour is derived from the maximum of the period 
signal, i.e. where the colour is not affected by the grid pattern. The relative depth of each 
image portion is calculated from the modulation contrast derived from either of the 
previous calculations. The actual depth is then calculated from a look up table obtained 
10 in a calibration step, which is simply an imaging operation as just described, compared 
with actual measurements of the distance of various portions of a test object from the 
imaging lens. 

Where more than one image, and/or more than one grid position are involved, these 
15 calculations are made for each image and grid position, as will be seen from the sub flow 
diagrams for Step 6 as shown in Figures 11, 12 and 13. Figure 13 has an option to use 
single grid or n grid depth extraction algorithms. 

Figures 14, 15, 16 and 17 are flow charts for exemplary imaging methods selected from 
20 the more generalised flow charts of the preceding figures. 

Many variations are possible within the context of the invention. Different methods may 
be used for illuminating the object, including filament lamps, fluorescent lamps, lasers 
and so on. It is possible to use single wavelength light, or even infrared or ultraviolet 

25 light, if colour is not required, and appropriate imaging devices are used. Instead of a 
'mechanical' grating, an electronic grating can be used, which can be controlled as to 
frequency and position. And different arrangements may be used for displaying and 
manipulating the final image, including a laser writing arrangement to a glass or plastic 
block or a computer assisted manufacturing arrangement which may involve spark 

30 erosion or other shaping technology, for rapid prototyping. 



